Dense video captioning based on local attention

Dense video captioning aims to locate multiple events in an untrimmed and generate captions for each event. Previous methods experienced difficulties establishing the multimodal feature relationship between frames captions, resulting low accuracy of generated captions. To address this problem, a novel Video Captioning Model Based on Local Attention (DVCL) is proposed. DVCL employs 2D temporal differential CNN extract features, followed by encoding using deformable transformer that establishes global dependence input sequence. Then DIoU TIoU are incorporated into event proposal match algorithm evaluation during training, yield more accurate proposals hence increase quality Furthermore, LSTM based local attention designed enabling word correspond relevant frame. Extensive experimental results demonstrate effectiveness DVCL. On ActivityNet Captions dataset, performs significantly better than other baselines, with improvements 5.6%, 8.2%, 15.8% over best baseline BLEU4, METEOR, CIDEr, respectively.